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Abstract. Let X = T\G/K be a compact locally symmetric space. In this paper we 
establish a version of the Selberg trace formula for non-unitary representations of the 
lattice r. On the spectral side appears the spectrum of the "flat Laplacian" A^^, acting 
in the space of sections of the associated flat bundle. In general, this is a non-self-adjoint 
operator. 



1. Introduction 

Let G be a connected real semisimple Lie group with finite center and of non-compact 
type. Let i^" be a maximal compact subgroup of G. Then S = G/K is a Riemannian sym- 
metric space of nonpositive curvature. We fix an invariant metric on S which we normalize 
using the Killing form. Let F C G be a discrete subgroup such that r\G is compact. For 
simplicity we assume that F is torsion free. Then F acts properly discontinuously on S 
and X = r\S is a compact locally symmetric manifold. 

Let r GL(K^) be a finite-dimensional unitary representation. Denote by r\S 
the associated fiat vector bundle. It is equipped with a canonical Hermitian fiber metric 
and a compatible fiat connection V^. Let d^: C°°{X, E^) — > A^{X,E^) be the associated 
exterior derivative and let S^. be the formal adjoint of with respect to the inner products 
in C°°{X,E^) and C°°(X, T*(X) (g) E^), respectively, induced by the invariant metric on 
5* and the fiber metric in E^. Let = S^d^ be the associated Laplace operator. Then 
is a second order elliptic, formally self-adjoint, nonnegative differential operator. The 
Selberg trace formula computes the distributional trace of costy^A^ in terms of a sum of 
distributions on G, which are associated to the conjugacy classes of F. 

The trace formula has many applications. Of particular interest for the present paper are 
applications to Ruelle and Selberg zeta functions. Especially the analytic continuation and 
the functional equation of twisted Ruelle and Selberg zeta functions rely on the twisted 
Selberg trace formula |B0] . |Se2] . Also spectral invariants of locally symmetric spaces 
such as analytic torsion and eta invariants can be studied with the help of the trace 
formula (see [Milj . [MSlj . |MS2] ). So far, these apphcations are restricted to unitary 
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representations of T and it this very desirable to extend the scope of the trace formula so 
that all finite-dimensional representations are covered. This is the main goal of this paper. 

To begin with we recall the trace formula for a unitary representation x (see |Selj . |Se2j ). 
Let spec(A;^,) be the spectrum of A^. It consists of a sequence < Ai < A2 < ■ ■ ■ of 
eigenvalues of finite multiplicities. Denote by m(Afc) the multiplicity of A^. Let ip G S(M.) 
be even and assume that the Fourier transform (p of (p belongs to C^(]R). Then y9((A^)^/^) 
is a trace class operator and its trace is given by 

(1.1) Try,((A,)V2)^ ^ m{XMX'/'). 

Agspec(A^) 

Let A be the Laplacian of S, and let h^^ be the kernel of the invariant integral operator 
<y9(A^/^). It belongs to the space C^{G//K) of i^'-bi- invariant compactly supported smooth 
functions on G. Given 7 G F let {7}r denote its F-conjugacy class. Furthermore, let 
and F^ denote the centralizer of 7 in G and F, respectively. Then the first version of the 
trace formula is the following identity. 

J2 m(A)(/?(A^/2) _ vol(F\5) dim V^h^ie) 

Aespec(Ax) 

(1-2) r 

+ 2^ trx(7)vol(F^\G^) / h^{g ^-fg) dg. 

To make this formula more explicite, one can use the Plancherel formula to express in 
terms of (p. Furthermore, the orbital integrals 

Ha;^) = / h^ig~^ig) dg 

JG-y\G 

are invariant distributions and therefore, one can use Harish-Chandra's Fourier inversion 
formula to compute them (see [DKVl §4] ) . In the higher rank case this is rather complicated 
and no closed formula is available. In the rank one case, however, the situation is much 
better. There is a simple formula expressing the orbital integrals in terms of characters 
which leads to an explicit form of the trace formula jWat Theorem 6.7]. 

To extend the Selberg trace formula to all finite-dimensional representations of F, we first 
note that the sum on the right hand side of (II. 2p is finite and therefore, it is well defined for 
all finite-dimensional representations x- The question is what is the appropriate operator 
which replaces the Laplacian on the left hand side. In general there is no Hermitian metric 
on which is compatible with the fiat connection V^. A special case has been studied by 
Fay )Faj . He considered the analytic torsion Tm{x) of a Riemann surface M = F\E[ of genus 
g > 1 and a unitary character x ^ Hom(F, S"^) and established the analytic continuation 
of Tm{x) to all characters x ^ Hom(F,C*). To this end he introduced a non-self-adjoint 
Laplacian. We use a similar approach in the general case. The operator that replaces A^ 
is the "fiat Laplacian" A* which is defined as follows. Let *: Ap{T*X) A"-P(r*X) 
be the Hodge star operator associated to the Riemannian metric of X. Extend * to an 
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operator *^ in Kp{T*X) E^hy = *® Id^^. Define 5* := *^ *^ . Then the 

fiat Laplacian A* is defined as 

A# = 5*d,. 

If X is unitary, A* equals A^. For an arbitrary x ^6 pick any Hermitian fiber metric in 
and use it together with the Riemannian metric on X to introduce an inner product 
in C°°(X, i?-^). In general, A# is a not self-adjoint w.r.t. this inner product. However, 
if we define the corresponding Laplace operator A^ as above by ^x'^xi where the formal 
adjoint 5^ is taken w.r.t. to the inner product, then A* has the same principal symbol 
as A^. This implies that the operator A* has nice spectral properties. Its spectrum is 
discrete and contained in a positive cone C C C with C C (see [SFJ). It follows that 

1 /2 

an Agmon angle 6 exists for A* and we can define ((A*)^ ) by the usual functional 

1 /2 

calculus [Sh]. This is a trace class operator. Since ip is assumed to be even, ((A*)^ ) is 
independent of 9 and we can delete 9 from the notation. Lidkii's theorem [GKt Theorem 
8.4] generalizes fll.ip . As mentioned above, the spectrum spec(A*) of A^ is discrete and 
consists of eigenvalues only. For A G spec(A*) let m(A) denote the algebraic multiplicity 
of A, i.e., m(A) is the dimension of the root space which consists of all / G C°°(X, i?^) such 
that there is G N with (A^ — A I)^/ = 0. Then by Lidskii's theorem we have 

(1.3) Tr^((A#)V2)= m(A)y.(AV2). 

Aespcc(Aj) 

The first version of our trace formula generalizes (II. 2p with Tr ip ((^*) ^^^) on the left hand 
side. 

Actually, we prove a more general result. Let r be an irreducible representation of 
K and Er T\S the associated locally homogeneous vector bundle, equipped with its 
canonical invariant connection V"^. Let V = V"^'^ be the product connection in E^ ^ E^, 
and let A*^ = — Tr(V^) be the corresponding connection Laplacian. Then for ip as above 
(p ((A*^)^/^) is a trace class operator and we establish a trace formula for this operator 
which is similar to the scalar case. 

If G has split rank one, we get an explicite version of the trace formula. To describe it 
we need to introduce some notation. Let G = KAN be an Iwasawa decomposition of G. 
Then dim A = 1. Let a be the Lie algebra of A. The restriction of the Killing form to a* 
defines an inner product on a*. Let \p\ denote the norm of the half-sum p of positive roots 
of (G, A). Let 7 G r \ {e}. Then there is a unique closed geodesic that corresponds to 
the F-conjugacy class {7}r of 7. Denote by /(7) the length of r^. Furthermore, let 70 G F 
be the unique primitive element such that 7 = 7o for some A; G N. Finally let .0(7) be the 
discriminant of 7 (see (16.21) for its definition). Let (3{X)dX be the Plancherel measure for 
spherical functions on G |He] . We can now state our main result in the scalar case which 
is the following theorem. 
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Theorem 1.1. For all even if G 5(]R) with (p G C^(]R) we have 

J2 rniX)^ ((A - = dMVy-^^^ [ ^(A)/3(A) dX 

4) Aespcc(A#) 

+ E trx(7)^^(/(7)). 

Note that for every c > there are only finitely many conjugacy classes {7}r with 
^(7) ^ c. Therefore the sum on the right hand side is finite. 

To describe our method we restrict attention to the scalar case, i.e, we consider the 
operator A*. Our method is based on the approach of Bunke and Olbrich |B0] to the 
Selberg trace formula in the unitary case. We consider the wave equation 

(1.5) + A#^ u{t) = 0, m(0) = /, urn = 0, 

for any initial conditions / G C°°{X,E^. Since the principal symbols of A-^# is given by 
(T(a;, ^) =11 ^ 11^ Id^^, the operator L = + A* is strictly hyperbohc in the sense of [Tail 
Chapt. IV, §3]. Therefore (16. 2p has a unique solution u(t; /). Let ip G S(M.) be even such 
that If G C^°°(M). Then it follows that 

(1.6) ^{i^ty^")f = ^ I mu{t-f)dt. 

Let u{t] f ) and / denote the lift of u{t, f ) and /, respectively, to the universal covering S of 
X. Then the corresponding wave equation on S with initial conditions u{0) = /, Ut{0) = 
is also strictly hyperbolic and by finite propagation speed it follows that it has a unique 
solution u{t] /). Thus we obtain u{t, x; /) = u{t, x; /). Since the lift of to S is trivial, 
the lifted operator A* takes the form A* = A Idy^ , where A is the Laplace operator 

on S. Let h^p G C^{G/ /K) be the kernel of the G-invariant integral operator (f ^A^^^^ 

Then it follows that the kernel K^^^x, y) of ((A*)-*^/^) is given by 

(1-7) K^{x, y) = ^ K{9i^l92)x{l), 

where x = TgiK and y = Tg2K. One can now proceed in the same way as in the case of 
a unitary representation x ^i^nd derive the twisted Selberg trace formula. 

Besides unitary representations of F, there is a second class of representations of F for 
which the usual trace formula can be applied. These are representations which are the 
restriction to F of a finite-dimensional representation rj: G ^ GL{E). Let E^^ ^ X 
be the flat vector bundle associated to 77 |r. Then E.r^ is canonically isomorphic to the 
locally homogeneous vector bundle Er associated to the principal i^-bundle V\G X via 
the representation r = tjIk- The bundle carries a canonical Hermitian fiber metric and 
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the Laplacian in C°°(X, i?^) with respect to this metric is closely related to the Casimir 
operator acting in C°°{X, E^). This brings us back to the usual framework of the Selberg 
trace formula for locally homogeneous vector bundles. Details will be discussed in section 

m 

The paper is organized as follows. In section [2] we collect a number of facts about 
spectral theory of elhptic operators with leading symbol of Laplace type and we develop 
some functional calculus for such operators. The kernels of the associated integral operators 
are studied in section [3l Especially, we prove (11. 6p and (11. 7p . In section H] we apply these 
results to the case of twisted Bochner-Laplace operators. In section Owe turn to the locally 
symmetric case and we prove the first version of the trace formula which is Theorem 15. 1[ 
In section |6] we specialize to the case where G has split rank one and we prove Theorem 11.11 
In the final section [7] we are concerned with representations of F which are the restriction 
of a representation of G. 

2. Functional calculus 

In this section we develop the necessary facts of the functional calculus we are going to 
use in this paper. 

Let X be a closed Riemannian manifold of dimension n and E X a, Hermitian vector 
bundle over X. We denote by C°°(X, i?) the space of smooth sections of E, and by 
L^(X, E) the space of L^-sections of E w.r.t. the metrics on X and E. Let 

P: C^{X,E) C^{X,E) 

be an elliptic differential operator of order 2 with leading symbol 

(2.1) a{P)ix,0=U\\l-ldE^. 
For / C [0, 27r] let 

= {re^^: < r < oo, 9 e I}. 

be the solid angle attached to /. The following lemma describes the structure of the 
spectrum of P. 

Lemma 2.1. For every < e < 7r/2 there exists R > such that the spectrum of P is 
contained in the set Br{0) U ^[-s,e]- Moreover the spectrum of P is discrete. 

Proof. The first statement follows from [Sh, Theorem 9.3]. The discreteness of the spectrum 
follows from [Shi Theorem 8.4]. □ 

It follows from Lemma 12.11 that there exists an Agmon angle 6 for P and we can define 
the square root Pg^'^ as in [Sh] . For the convenience of the reader we include some details. 
Denote by spec(P) the spectrum of P. Let e > 0. For simplicity we assume that ^ cr(P). 
By Lemma 12.11 there exist < 6 < 2it and e > such that 

spec(P) n A[e-e,e+e] = 0- 
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6 is called an Agmon angle for P. Since spec(P) is discrete and ^ <^{P), there exists also 
To > such that 

spec(P) n G C: 1^1 < 2ro} = 0. 
Define the contour F = Tg^ro C C as the union of three curves F = Fi U r2 U F3, where 

Fi = {re*^: tq < r < 00}, F2 = {roe^": e<a < ^ + 27r}, 
F3 = {re^(^+27r). < ^ < 00}. 

Put 

By [Shi Corollary 9.2, Chapt. II, §9] we have || (P-A)-i ||< C\X\-^ for A G Te,ro- Therefore 
the integral is absolutely convergent. Put 

Then Pg^'^ satisfies (Pj^^)^ = P. U 6 is fixed, we simply denote this operator by P^/^. We 
recall |See] . |Shl Theorem 11.2] that P^/^ is a classical pseudo-differential operator with 
principal symbol 

a{P'/'){x,0=Uh-ldE.. 
In any coordinate chart, the complete symbol q{x,^) of P^^^ has an asymptotic expansion 

00 

i=o 

where ^) is homogeneous in ^ of order I — j. The same holds for A]/^. Since the 

principal symbols coincide, we get 

(2.2) P^/2 _ ^1/2 ^ 

where P is a pseudo-differential operator of order zero. Especially, P is a bounded operator 
in L2(X,P). 

Let Rx{P^'^) = (P^/^ - AI)"i and Pa(A]/^) = (A^^^ - AI)-i be the resolvents of P^/^ 

1 /2 1 /2 

and A^ , respectively. For A ^ spec(A^ ) we have the following equality 

(2.3) pi/2 _ ^ J ^ (I+PP,(A^/'))(A^/' - A I). 
Since A^J'^ is self-adjoint, the resolvent of A]/^ satisfies 

(2.4) ||P,(Af)||<|Im(A)ri 

pCal Chapt. V, §3.5]. Let 6 = 2 || P ||. It follows from ([MD that for | Im(A)| > 6 we have 
II PPa(A]/^) II < 1/2. Thus in this range of A the operator l+BRx{A^^^) is invertible and 

Pa(P'/') = RxiAf){l+BR,{Af))-\ 
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Combined with (12.41) we get 

(2.5) \\ Rx{P^^^) \\< 2\lm{X)\~\ |Im(A)|>6. 

We can now summarize the spectral properties of P^/^. 

Lemma 2.2. The resolvent of P^^"^ is compact. The spectrum of P^^^ is discrete. There 
exist b > and c G M such that the spectrum of P^/^ is contained in the domain 

(2.6) = {A G C: Re(A) > c, |Im(A)|<6}. 



Proof. Since P^/^ is an elhptic pseudo-differential operator of order 1 on a closed manifold, 
its resolvent is compact and hence, its spectrum is discrete. The remaining statements are 
a consequence of (12.51) . □ 

Though P is not self-adjoint in general, it still has nice spectral properties |Sh^ Chapt. 
I, §8]. Given Aq G spec(P), let F^o be a small circle around Aq which contains no other 
points of spec(P). Put 

Hao = f / RxiP) dX. 

Then 11^0 is the projection onto the root subspace Vaq. This is a finite-dimensional subspace 
of C°°(X, E) which is invariant under P and there exists N & N such that (P — Aq I)^Vao ~ 

0. Furthermore, there is a closed complementary subspace V^^ to Va,, in L'^{X,E) which 
is invariant under the closure P of P in and the restriction of (P — Aq I) to V^^ has a 
bounded inverse. The algebraic multiplicity m(Ao) of Aq is defined as 

m(Ao) = dimVxo- 

If Ai, A2 G spec(P) with Ai 7^ A2, then the projections FIai and FlAa are disjoint, i.e., 

Ux.Ux, = Ux.Ux, = 0. 

Since the principal symbols of P and A^; are the same, we have P = A + D, where D is 
a first order differential operator. Let Pa (A) be the resolvent of A. Then DRx{A) is a 
compact operator. This means that D is compact relative to A and it follows from [Mkl 

1, §4, Theorem 4.3] that the root vectors are complete. Thus L^(X, E) is the closure of the 
algebraic direct sum of finite-dimensional P-invariant subspaces Vk 

(2.7) L2(X,P) = 014 

k>l 

such that the restriction of P to has a unique eigenvalue A^ and |Afc| 00. In general, 
the sum (12. 7p is not a sum of mutually orthogonal subspaces. 

It follows from (12. 7p that Pg^'^ has a similar spectral decomposition with eigenvalues A^^, 

1 /2 

A G spec(P), and ra{\J ) = m(A). 
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Given r > 0, let 

N{r,P):= J2 

Aespec(P), |A|<r 

be the counting function of the eigenvalues of P, where eigenvalues are counted with their 
algebraic multiplicity. 

Lemma 2.3. Let n = dimX. We have 

Proof. It is well know that Tr(e~*^^) has an asymptotic expansion of the form 

Tr(e-*^^) ~r"/2^afc^^ t ^ +0 

k>0 

(see [Gil Lemma 1.8.3]), and by |Gil Lemma 4.1.4] the leading coefficient is given by 
= (47r)~"/^ rk(i?) vol(X). Let X(r, Ag) be the counting function of the eigenvalues of 
A^;. Using the Tauberian theorem (see |Shl Chapt. II, §14]), we get 

The lemma follows from |Mkt I, §8, Corollary 8.5]. □ 

Let h G C;?°(M) be even and set 

<^{X) = — ^ / h{r) cos(tA) dr, A G C. 

Then (f{X) is rapidly decreasing in each strip | Im(A)| < 6, 6 > 0. For 6 > and (i G M let 
r = C C be the contour which is union of the two half-lines L±f„d = {z E C: lm{z) = 
±b, Re(z) > d} and the semi-circle S = {d + be'^: ti/2<9 < 37r/2}.' By LemmaOthere 
exist 6 > 0, d G M such that spec(P^/^) is contained in the interior of T^.d- Put 

(2.9) ^(pi/2) := f / ^iX)iP'/' - XiyUX. 

It follows from (12. 5p that the integral is absolutely convergent. 
Lemma 2.4. LpiyP^/"^) is an integral operator with a smooth kernel. 

Proof. First note that 

¥^(A)(P - A2)(P^/' - Xy' dX= [ ip{X){P^/^ + X)dX = 0. 

Jr 

This implies that for /c, / G N we have 

pV(P'/')P' = — / A2('=+'V(A)(P'/'-A)-idA. 
2tc Jr 
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The function A t-^ A^*^'^"'''V('^) is rapidly decreasing on | Im(A)| = ±b. Hence P'^(p{P^^'^)P^ 
is a bounded operator in L^. Since P is elliptic, it follows that for all s, r G M, (p{P^^'^) 
extends to a bounded operator from H^{X,E) to H'^{X,E), which shows that y9(P^/^) is 
a smoothing operator and hence, it is an integral operator with a smooth kernel. □ 

In order to continue, we need to establish an auxiliary result about smoothing operators. 
Let 

be an integral operator with a smooth kernel H G C°°{X x X,EM E*). 
Proposition 2.5. A is a trace class operator and 

(2.10) Tt{A)= / ti H{x,x) dfi{x). 

Jx 

Proof. We generalize the proof of Theorem 1 in |La[ Chapt VII, §1]. Let be a Hermitian 
connection in E and let A^; = (V^)*V^ be the associated Bochner-Laplace operator. Then 
A^; is a second order elliptic operator which is essentially self-adjoint and non- negative. 
Its spectrum is discrete. Let {0j}jeN be an orthonormal basis of L'^{X,E) consisting of 
eigensections of A^; with eigenvalues < Ai < A2 < ■ ■ ■ ^ C)0. We can expand H in the 
orthonormal basis as 

00 

(2.11) H{x,y) = ^ aij(f),{x) ® (f)*{y), 
where 

(2.12) a,j = {A4>i,(Pj). 

Since H is smooth, the coefficients rapidly decreasing. Indeed, for every we have 

(l + Ai + A,-)Vj = {{l+AE(E)l + l(E)AEfH,(f),^(P*). 

Hence for every G N there exists > such that 

Iflijl < C'7v(l + Ai + Aj)"^, i,jeN. 

This implies that the series 02.111) converges in the topology. Let Pij be the integral 
operator with kernel (pi® cf)*. Thus 




k = j. 



Let Pj be the orthogonal projection of L'^{X, E) onto the 1-dimensional subspace C(/)j. Put 
B = ^a,,(l + A,)"P,„ C = ^(1 + A,)-"P,. 
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Then A = BC and it follows from (12.81) that B and C are Hilbert-Schmidt operators. Thus 
A is a trace class operator. Furthermore, by (12. lip and (I2.12p we get 

« oo « oo 

/ iiH{x,x) dx = 2^ o^i j / dx = y^aa = Tt{A). 

u=i ' i=l 



□ 



Now we apply this result to Lp{P^^'^). Let K^(x,y) be the kernel of Lp{P^^'^). Then by 
Proposition 12.51 ip{P^^'^) is a trace class operator and we have 

(2.13) Tt<^{P^/^)= / tr K^{x,x) dfi{x). 

Jx 

By Lidskii's theorem \GK\ Theorem 8.4] the trace is equal to the sum of the eigenvalues of 
(y9(P^/^), counted with their algebraic multiplicities. The eigenvalues of ip{P^^'^) and their 
algebraic multiplicities can be determined as follows. Given G N, let IIjv denote the 
projection onto the direct sum of the root subspaces Vk, k < N, of P. As explained above, 
we have 

N 

PU^ = Y,ih^k + D,), 

k=l 

where 11^ is the projection onto Vk and Dk is a nilpotent operator in Vk. Then it follows 
from [Kil I, (5.50)] that 

TV 

(^(pi/^)n^ = ^(^(Af)n, + D^), 

k=l 

where is again a nilpotent operator in Vk- Thus Lp{P^^'^) leaves the decomposition (12. 7p 
invariant and the restriction of V9(P^/^) to Vk has a unique eigenvalue (p{X]/'^). Of course, 

1 /2 

some of the eigenvalues v^(A^ ) may coincide in which case the root space is the sum of 
the corresponding root spaces Vk- Now, applying Lidskii's theorem [GK\ Theorem 8.4] and 
(I2.13p . we get the following proposition. 

Proposition 2.6. Let ip G S{R) be even and suppose that (p G C^(]R). Then we have 

(2.14) ^ m(A)v?(Ai/2) = I tiK^{x,x)dx. 

AGspec(P) 

By Lemma [2. 3 [ the series on the left hand side is absolutely convergent. 
Remark. Recall that P^^^ = Pg^"^ depends on the choice of an Agmon angle 6, and so do 

1 /2 1 /2 

the eigenvalues A^ = {\k)g ■ Let < ^ < 6*' < 27r be two Agmon angles. Then it follows 
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from Lemma [2.11 that there are only finitely many eigenvalues Ai,...,Am of P which are 
contained in Aw 9/1. Therefore for A G spec(P) we have 



Wo'' 



1/2 



(A)f, if A^{Ai,...,A^}; 
-(A)^', if Ae{Ai,...,A^}. 



Since ip is even ip ^(A)^^ j is independent of 9. This justifies the notation on the left hand 
side of (EUD. □ 

3. The kernel and the wave equation 

In this section we give a description of the kernel of the smoothing operator (y9(P^/^) 
in terms of the solution of the wave equation. Consider the wave equation 

(3.1) — + Pu = 0, uiO,x) = fix), uti0,x)=0. 

Proposition 3.1. For each f G C°°{X,E) there is a unique solution u{t; f) G C°°(M x 
X,E) of the wave equation 1^3. 1\) with initial condition f. Moreover for every T > and 
s G M there exists C > such that for every f G C°°{X, E) 

(3.2) \\u{tJ)\l<C\\u{QJ)\U |t|<T, 
where || • \\s denotes the s-Soholev norm. 

Proof. We proceed in the same way as in [Tail Chapt. IV, §§1,2] and replace (13.11) by a 
first oder system. Let A^; be the Bochner-Laplace operator associated to the connection 
in E. Put A = (As + Idfl^ and 

Q C^iX,E) C^iX,E) 

-PA-i ) ■ ® ' © • 

/ C°°{X,E) C^{X,E) 

Then L is a pseudo-differential operator of order 1. Let m be a solution of (13.11) . Put 



(3.3) L :-- 
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ui = Am, U2 = ^u. 

at 



Then {ui,U2) satisfies 



d 



On the other hand, let (mi,M2) be a solution of the initial value problem (13. 4p . Put 
u = A^^ui. Then m is a solution of (13.11) . Thus it suffices to consider (13.41) . By (12. ip 
it follows that P = Ae + D where D is a differential operator of order < 1 . Therefore we 
get 

PA-^ = {Ae + Id)A-i + iD- ld)A-^ = A + Pi, 
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where Bi is a pseudo-dfifferential operator of order 0. Therefore 

/ A\ [0 -A-B*\_ [0 El 
^ + ^ - ^-A - 5i J + )-~\B, Q 

is a pseudo-differential operator of order zero. Hence (13 ■4p is a symmetric hyperbohc system 
in the sense of [Talj Chapt. IV, §2]. So we can proceed as in the proof of Theorem 2.3 
in [Talt Chapt. IV, §2] to establish existence and uniqueness of solutions of (13. ip . The 
estimation (13.20 follows from the proof using Cronwall's inequality. □ 

Proposition 3.2. Let ip G 5(]R) he even such that (p G C^(]R). Then for every f G 
C°^{X,E) we have 

y,(pV2)j= 1 / muit;f)dt. 
y 2.TX Jr 



Proof. Let F C C be as in (12.90 . Let c > be such that the spectrum of P + c is contained 
in Re(z) > 0. For cr > define the operator cos(tP^/^)e~°'^^^'^'' by the functional integral 

cos(tpi/2)g-<x{p+c) _ ^ f cos{tX)e-''^^'+'\P^/^ - X)-UX. 



r 



27T 

By (12.50 the integral is absolutely convergent. For / G C°°(X, E) and a > put 
(3.5) u{t; a, f) := cos(tP^/2)g-<x(P+c)^_ 

Then u{t; cr, /) satisfies 

52 \ , I 



+ PJ u{t- ^, /) = ^ cos(tA)e-'^(^ +^)(P - \^){P'" - X)-' dX 



— [ cos(tA)e-'^(^'+^)(pi/' + A)c/A = 0. 
27r Jr 



and u{0;a,f) = e-^^^^''^ f . Thus u{t;a,f) is the unique solution of (13.11) with initial 
condition e~"'^^~^'^^ f . Then u{t; f) — ^(t; a, f) is the solution of (13.10 with initial condition 
/ - e-(^+^)/. Hence by we get for all s G M 

(3.6) II uit; f) - u{t- a, /) ||h=< C || / - e-'^(^+^)/ ||^., |t| < T. 
Now note that for every / G C°° (X, i?) we have 

lim II e""(^+'V - / 11= 0. 
This follows from the parametrix construction. Hence we get 

II / _ e-(^+^)/ 11^. = II (P + cy/'f - e-'^(^+^)(P + cyi'f lu.^ 
as 0" — i> 0. Combined with (13. 6p we get 

(3.7) lim \\u{t-f)-u{t-aj) ||h== 0. 



j^^{X)e-''^'"+''\P'/' - X)-'f dX. 
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Furthermore we have 

/ mu{t;a,f)dt = ^ I (p{t)^ I cos{tX)e-'^^''^^\P'/'-X)-'fdXdt 
V27r Jr V27r Jr 2,71 

= — f ( [ ^{t) cos(tA) dt] e-(^'+-)(pi/2 _ xy^fdX 
27r Jr V V2tt Jk J 
i 

For a ^ 0, the right hand side converges to Lp{P^/'^)f . By (13.71) the left hand side converges 
to(27r)-V2J^^(t)«(t;/)rft. □ 

Let p: X X be the universal covering oiX,E = p*E,jindP: C°°(X, E) C~(X, E) 
the lift of P to X. Let u(t, x, /) and / be the pull back to X of uit, x; f ) and /, respectively. 
Then u{t, f) satisfies 

(3.8) (^:^ + p^u{t;f) = 0, u{0-J) = lu,{OJ)=0. 

By (12. ip we have P = Ae + D, where D is a differential operator of order < 1. Then it 
follows from energy estimates as in |Ta2t Chapt. 2, §8] that solutions of {■^ + P)u = have 
finite propagation speed. This implies that for every ip G C°°{X, E) the wave equation 

— + P u{t- ^) = 0, u{0- ^) = ij, ut{0; ij) = 0, 



has a unique solution. Hence we get 
(3.9) u{t;f) = u{tj). 

Let d{x, y) denote the geodesic distance of x, y G X. For 5 > let 

U& = {{x,y) G X X X: d{x,y) < 6}. 

Proposition 3.3. There exist 6 > and G C°°(X xX, Hom(£', E)) with supp C Us 
such that for all ip G C°°(X, E) we have 

— ^ / ^{t)u{t,x;ip) dt = l__H^{x,y){ip{y)) dy. 
V 2tt Jr J X 

Proof. Suppose that supp^ C [— T, T]. Let V C X be an open relatively compact subset. 
For r > let 

Vr = {yeV:d{y,V)<r}. 

Let X £ C'^{y2T) such that x = 1 on ^t- By finite propagation speed, we have 

u(t,x;ip) = u(t,x;xi'): x & V, |t| < T, 

for all ip G C°°{X , E). Thus we are reduced to the case of a compact manifold and the 
proof follows from Lemma 12.41 and Proposition 13. 2[ □ 
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Using Proposition 13.31 together with fl3.9p and Proposition 13.21 we obtain 



(3.10) ^iP'^')fix) = [H^ixMfiy)) dy 



J X 



for all / G C°°{X^E). Let F d X he a. fundamental domain for F. Given 7 G P, 
\ei \ E ^ E be the induced bundle map. Thus for each y G X, we have a linear 
isomorphism : Ey E^y. Note that / satisfies 



Combining this expression with fl3.10p . it follows that the kernel K^p of ip{P^^'^) is given by 



7Gr 

where x and y are any lifts of x and y to the fundamental domain F. So by Proposition 
12.61 we get 

Proposition 3.4. Let (f G S(R) be even and suppose that (p G C^(]R). Then we have 



Note that the sum on the right is finite. 

4. The twisted Bochner-Laplace operator 

Let £■ ^ X be a complex vector bundle with covariant derivative V. Define the invariant 
second covariant derivative by 

where U, V are any two vector fields on X. Then the connection Laplacian A* is defined 



f{iy)=RMiy)), 7eP. 



Then we get 




(3.11) 





by 



A* 



Tr(V2). 
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Let (ei, e„) be a local frame field. Then 

i 

This formula implies that the principal symbol of A* is given by 

a{A*){x,0=U\\lldE.- 
Thus the results of the previous section can be applied to A*. 

Assume that E is equipped with a Hermitian fiber metric and V is compatible with the 
metric. Then it follows that 

(4.1) V*V = -TrV^ 

[LMt p. 154], i.e., the connection Laplacian equals the Bochner-Laplace operator A^; = 
V*V. 

Now let p: vri(X) GL(V^) be a finite-dimensional complex representation of tti{X). Let 
F X he the associated fiat vector bundle with connection V^. Let E he a Hermitian 
vector bundle with Hermitian connection V^. We equip E®F with the product connection 
Y_B®F^ "which is defined by 

Vp®^ = V^®1 + 1®V^, 

for Y G C°°(X, TX). Let A*^ be the connection Laplacian associated to y^®^ _ Locally 
it can be described as follows. Let U (Z X he an open subset such that F\u is trivial. Then 
{E ® F)\u is isomorphic to the direct sum of m = rank(F) copies of E\u: 

{E®F)\u = ®T=iE\u. 

Let ei, Cm he a basis of fiat sections of F\u. Then each ip G C°°{U, {E F)\u) can be 
written as 

m 

i=i 

where ipi G C°°{U, E\u), i = l,...,m. Then 

j 

Let A^; = (V^)*V^ be the Bochner-Laplace operator associated to V^. Using (14. ip . we 

get 

(4.2) A|^(^ = ^(A^<^,)®e,. 

j 

Let E and F he the pullback to X of and F, respectively. Then F = X x V and 

C°°(X, E^F) = C°^(X, E) ® V. 
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It follows from (14 ■2p that with respect to this isomorphism, the lift A* ^ of A* ^ to X takes 
the form 

A| ^ = Ae® Id, 

where A^; is the lift of A^; to X. Let tjj G C^{X, E) V. Then the unique solution of the 
wave equation 



^ + A| uit; ^) = 0, m(0; ^j) = ^, ^^(0, V^) = 0, 
is given by 

u{t; ^) = (^cos (t(As)i/2 j ® idj ^. 
Let ip be as above and let k^p{x,y) be the kernel of 

cp ((A^)V2) = -^jjit) cos (t(A^)V2) 

Then the kernel of Proposition 13.31 is given by H^(x, y) = k^ix^y) ® Id. Let R^: Ey ^ 
Eyy be the canonical isomorphism. Then it follows from (13.111) that the kernel of the 
operator (y9((Af ^p)^/^) is given by 

(4.3) K^{x, y) = J2 Ki^^ ° (^7 ® P(7))- 

7er 

Combined with (13.41) we get 

Proposition 4.1. Let Fp he a flat vector bundle over X , associated to a finite- dimensional 
complex representation p: tti{X) GL{V). Let A*^ be the twisted connection Laplacian 
acting in C°°{X,E (g) Fp). Let ip G S{R) be even with (p G C^(R) and denote by k^{x,y) 

the kernel of (p ( {AeY^'^ ) ■ Then we have 



(4.4) V m(A)(/7(A^/^) = Vtrp(7) / tr(A;<^(x,7x) oi?^) c?x. 



5. Locally symmetric spaces 



In this section we specialize to the case where X is a locally symmetric manifold. Let 
G be a connected semisimple real Lie group of non-compact type with finite center. Let 
C G be a maximal compact subgroup of G. Denote by q and t the Lie algebras of G 
and K, respectively. Let 

(5.1) g = P©e 

be the Cartan decomposition. Put S = G/K. This is a Riemannian symmetric space of 
non-positive curvature. The invariant metric is obtained by translation of the restriction 
of the Killing form to p = Te{G/K). Let F C G be a discrete, torsion free, cocompact 



17 

subgroup. Then T acts freely on S by isometries and X = T\S is a compact locally 
symmetric manifold. 

Let t: K —>■ GL(V^) be a finite-dimensional unitary representation of K, and let 

E^ = {Gx Vr)/K ^ G/K 

be the associated homogeneous vector bundle, where K acts on the right as usual by 

{g, v)k = {gk, T{k-^)v), g e G, k e K, v e Vr. 

Let 

(5.2) G^iG; r):={f:G^Vr\fe figk) = T{k-^)f{g), geG.keK]. 

Similarly, by G^{G; r) we denote the subspace of G°°{G; r) of compactly supported func- 
tions and by L'^{G; r) the completion of G^{G; r) with respect to the inner product 



(/i,/2)= / {fi{9)j2{9))dg. 

JG/K 

There is a canonical isomorphism 

(5.3) G^{S,Er) = G^{G]T). 



pn p.4]. Similarly, there are isomorphisms G^{S,E^) = C^{G]t) and L^{S,Er) 
L\G-t). 

Let be the canonical G- invariant connection on E^. It is defined by 

d 



(gexpitY))-' f(geMtY)K). 



t=Q 



where / G G°°{G;t) and F G p. Let be the associated Bochner-Laplace operator. 
Then is G-invariant, i.e., A^ commutes with the right action of G on G°°{S, Et-). Let 
fl G Z{qc) and Qk £ Z{tc) be the Casimir elements of G and K, respectively. Assume 
that r is irreducible. Then with respect to (15.31) . we have 

(5.4) Ar = -R{n) + Xrld, 

where At = t{Qk) is the Casimir eigenvalue of r |Mit Proposition 1.1]. We note that 
Xr > 0. 

Let (p G S(K) be even with G G^(]R). Then (p{Al-^'^) is a G-invariant integral operator. 
Therefore its kernel k^ satisfies 

Kiax, gy) = k^{x, y), g EG. 

With respect to the isomorphism (15.31) it can be identified with a compactly supported 
G°°-function 

h^: G^ End(V;), 

which satisfies 

K{kigk2) = T{ki) o h^{g) o r(/c2), ki, k2 G K. 
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Then (/^(Ar ) acts by convolution 

(5.5) (^(Ay^)/) (^7i) = / K{9l^92){f{g2))dg^. 

•J G 

Let 

Er = T\Er 

be the locally homogeneous vector bundle over T\S induced by Er- Let x- ^ ^ GL(V^) be 
a finite-dimensional complex representation and let E^ be the associated fiat vector bundle 
over T\S. Let A*^ be the twisted connection Laplacian acting in C°°(T\S, E^-^E^). Then 

it follows from (14. 3p that the kernel K^p of y(Ar ) is given by 

K^{g,K,g2K) = J2h^i9l'l92) ® xil)- 

By Proposition 14.11 we get 

(5.6) Try, ((A*^)^^) = J^^rxil) [ tr h^{g-'^g) dg. 

We now proceed in the usual way, grouping terms together into conjugacy classes. Given 
7 G r, denote by {7}r, 1^7, and the F-conjugacy class of 7, the centralizer of 7 in F, 
and the centralizer of 7 in G, respectively. With the conjugacy class {e}r separated from 
the others as usual, we get a first version of the trace formula. 

Proposition 5.1. For all even (f G 5(]R) with if G C^(]R) we have 
Tr y, ((A#^)V2) = dim{V^) vol(F\S) tr h^{e) 



(5.7) 

{7}r7^e 



+ ^^x{l)vo\(T^\G^) / tr h^{g ^jg) dg. 



In order to make this formula more explicit, one needs to express the kernel in terms 
of (fi, and to evaluate the orbital integrals on the right hand side. The kernel hip can be 
determined using Harish-Chandra's Plancherel formula. The orbital integrals can be com- 
puted using the Fourier inversion formula. However, both formulae are pretty complicated 
in the higher rank case. A sufficiently explicit formula can be obtained in the rank one 
case which we discuss in the next section. 



6. The rank one case 



Let G and K be as above. We introduce some notation following [WaJ. Let G = KAN 
be an Iwasawa decomposition of G (see [He]). Then A is a maximal vector subgroup of G 
and iV is a maximal unipotent subgroup of G. In this section we assume that G has split 
rank one, i.e., dim A = 1. Let M be the centralizer of A in K. We set P = MAN. Then 
P is a parabolic subgroup of G. Since G has split rank 1, every proper parabolic subgroup 
of G is conjugate to P. 
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Denote by G and M the set of equivalence classes of irreducible unitary representations of 
G and M, respectively. For tt G G we denote by ?i,r the Hilbert space in which vr operates. 

Let a and n be the Lie algebras of A and A^, respectively. Choose H E a such that 
ad(if)|n has eigenvalues 1 and possibly 2. Then a = RH. For t E R we set at = exp{tH) 
and loga* = t. Let A'^ = {at: t > 0}. 

Let p be the half-sum of positive roots of (g, a). Its norm \p\ with respect to the normalized 
Killing form is given as follows. Let p and q be the dimensions of the eigenspaces of ad(if) |n 
with eigenvalues 1 and 2, respectively. Then p > and < g < p. Then 

(6.1) H = i(p + 2g). 

For 0" G M and A G M let tTo- a be the unitarily induced representation from P to G which 
is defined as in \Wa\ p. 177]. Let Q„x denote the character of tTq-^a- 

If 7 G r, 7 7^ e, then there exists g E G such that g'jg'^ G MA^. Thus there are G M 
and a^ G A^ such that g'yg~^ = m^a^. By |Wal Lemma 6.6], a^ depends only on 7 and 
is determined by 7 up to conjugacy in M. Let 

/(7) = loga^. 

Then ^(7) is the length of the unique closed geodesic of r\S determined by {7}r- Further- 
more, by the above remark 

(6.2) D{-f) :=e-'(^)l^l|det(Ad(m^a^)|n-Id) | 
is well defined. Let 

m(7) = V0\{Gm^a^/A). 



Let h G G^{G) be iT-finite. Then by [Wal pp. 177-178] (correcting a misprint) we have 

(6.3) = I e,,(M . e-'«^ .A. 

Since h is ii'-finite, Qcr,\{h) 7^ only for finitely many cr. Thus the sum over a G M is 
finite. The volume factors in (15. 7p are computed as follows. Since G has rank one, is 
infinite cyclic |DKVl Proposition 5.16]. Thus there is 70 G F^ such that 70 generates F^ 
and 7 = 70^'*^'* for some integer n{'~f) > 1. Then 

(6.4) '-^^<^ = /(-,„). 

Inserting (16.31) and (16. 4p into (15. 7p . we get the following form of the trace formula in the 
rank one case. 
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Proposition 6.1. Let if e he even with ^ G C^°°(K). Then 

Tr^ ((A#^)'/') =dim(\/J vol(r\5)tr/i^(e) 
(6-5) ^ y trx(7)-^ 



The right hand side is still not in an explicite form. First of all we can use the Plancherel 
formula pCn] to express ii h^p{e) in terms of characters. In this way we are reduced to the 
computation of the characters G^, tt G G, evaluated on tr/i^. This is our next goal. 

For simplicity we assume that K is multiplicity free in G, i.e., for each t G K and 
TT G G, we have [7i\k '■ t] < 1. By |Ko] this condition is satisfied for G = 5'0o(n, 1) and 
G = SU{n,l). Let 



G(r) = {7rGG: [vrU : r] = l} . 



Then for each vr G G(r) we can identify the r-isotypical subspace 'Htt{t) of r in Ti^^ with 
Vr- Let Pr be the orthogonal projection of 7i.„ onto 7-^^(r). Define the r-spherical function 
$^ on G by 

^l{g) := PrT^{g)Pr. g E G. 

Then $^ is a C°°-map 

G ^ End(K) 

which satisfies 

^ ■ ^ $^(A;i^fc2) = T{h)<i>-^{g)T{h), geG, h,heK. 

Let f G K- and set 

(6.7) 

Then f^^^ G G°°(G; r) and it follows from ([53D that 

(6.8) A./-. = (-7r(r]) + A.)/-,. 

Let u{t,x; f^^) be the unique solution of 

Lemma 6.2. For t G M, r G and n G G(r), we have — 7r(fi) + At- > and 

x; fl,) = cos (tv/-vr(fi) + A,) r^x). 



+ A, ) n(t) = 0, n(0) = um = 0. 



x{y) 
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Proof. Let (■, ■) be the Killing form on q. Its restriction to p (resp. 6) is positive (resp. 
negative) definite. Let Xi,...,Xii G p and Yi,...,Ym G t be bases of p and t, respectively, 
such that {Xi, X,) = 6i„ {Y„ Yj) = -5i,. Then = - and = - ■ 

Let V G Hnir), \\ v \\= 1. Then we get 

+ Xr = -{7T{n)v,v) + A, = ^ II 7T{Xi)v f > 0, 

i 

which proves the first statement. For the second statement, we note that by definition, we 
have 

(6.9) g^u{t, X- fl,) = -AMt, X- fl,). 

Fix xq G S. Let x e C^i.^) be such that 

1, ye B2t{x); 
0, yeS\B,tix). 

Then by finite propagation speed we have 

uit, x; fl^) = u{t, x; xfr,v)^ ^ ^ Bt{xo). 
Since xfr,v e C^iS,E^), we have 

u{t,x; fl^) = (cos (i(A.)^/^) {xf:,j) (x), X G i?,(xo). 
Using that A,- commutes with cos (t(A^)i/2j, and finite propagation speed, we get 

AMt,x-J:^^) = uit,x;AJ^^J. 

By del]) it follows that 

u{t, X- A^fl,) = -(-vr(fi) + K)uit, X- fl^). 

Combined with (16. 9p it follows that for every x G S', u(t,x;f^^) satisfies the following 
differential equation in t 

^ - n{n) + u{t,x; fl^) = 0, n(0,x; f^^) = fl^{x), ni(0,x; (t>x) = 0- 

This implies the claimed equality. □ 

Let if G iS(]R) be even with (p G C^(R). Since the kernel of the integral operator 
(^((A,)i/2) is given by G C^{G), (^((A,)i/2)(/-J is well defined and it follows from 
Lemma 16.21 that 



^((Aoi/2)(/-j = ^ / ^{t)u{t- /-J dt = -= I m cos(tv'^^KW+A;)/.-. dt 
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If we rewrite this equality in terms of the kernel h^p and use the definition of /^^, we get 

(6.10) j^h^{g-'g,)^U9i')v dg, = ^ (^^Z^^pTA;) <^U9'>- 
Let cIt- := dimy^. Putting g = I and taking the trace of both sides, we get 

jjT[h^{g)^l{g-^)] dg = d^if {^-T^m + A.) . 
We continue by rewriting the left hand side. To this end put 

Note that 0^ satisfies 0^(5') = (p'!^{g~^). Using the Schur orthogonality relations (see |Knl 
Chapt. I, §5]), we get 

(6.11) <^Ug) = dr [ TT[T{k-')<i>UgMk) dk = dr [ ^^k-' g)T{k) dk. 
Using (16.111) . we get 

Tr[hM^;{g-')] dg = dr [ [ <Pl{k-^g-^)i:i[K{g)r{k)] dkdg 

= dr / (j)l{gk)iih^{gk) dgdk 
Jk Jg 



G 



(6.12) 



dr / tTh^{g)(f)^{g) dg. 



G 



Together with fl6.10p we obtain 



(6.13) 



G 



tr h^igWrig) dg = ^ ( v^^^^MTa;) . 



Now let r' G -ft' be any other representation which occurs in ti\k- Repeating the argument 
used in (16.121) . we get 



L 



K 



^rh^{g)(f>r'ig) dg= Tt 

• Jg 
Again by the Schur orthogonality relations, we have 

/ (t)l,{k-^g-^)T{k) dk = 

JK 

a t' ^ T. Hence we get 



rAk-'g-'Mk) dk hM 



dg. 



(6.14) 



ti h^{g)rAg)dg = 0, T'eK, r'^T. 



g 
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Choose an ortho normal basis of Ti^r which is adapted to the decomposition of 7r|x into 
irreducible representations of K. Then it follows from (16.141) that 

Q^iiih^) = Tr 

(6.15) 

= / ti h^ig)rr{9)dg. 
Jg 

Combined with (I6.13P we obtain the following lemma. 

Proposition 6.3. Let Lp G S(M.) be even with (p G C^(M). Let he the kernel of 
(/9((A^)^/^). Then for all n G G{r) we have 

Q^itih^) = ip (^V-7r(l^) + A,) . 

Since G has split rank one, the tempered dual of G (which is the support of the Plancherel 
measure) is the union of the unitarily induced representations n^^x, o" ^ M, A G M, and the 
discrete series, where the latter exists only if rank G = rank K. First consider the induced 
representation tCct^x- Let T C M be a maximal torus and t the Lie algebra of T. Let Ao- G it 
be the infinitesimal character of a G M and pm the half-sum of positive roots of {M,T). 
Then by [Knl Proposition 8.22] 

(6.16) 7r.,A(^^) = -A' - IpP + |A. + Pa/|' - |pa/|', 

where |p| is given by (16. ip . Let t G K. By Frobenius reciprocity |Knl p. 208] we have 

(6.17) : r] = [t\m : a], a G M. 

Since we are assuming that K is multiplicity free in G, it follows that [t\k : o"] < 1. Let 

M(r) = {aeM: [r\M : a] = 1}. 

Then by (16.170 it follows that tTo-.a G G{t) if and only if a G M(r), and by Proposition 16.31 
we get 

(6.18) e,,A(tr/i^) = ip + |p|2 + \p,j\^ - |A, + pm|2 + A,) , a G M(r), A G M. 

Now suppose that rank G = rank K. Then G has a non-empty discrete series. Let H G G 
be a compact Cartan subgroup with Lie algebra f). Let L C if) be the lattice of all p G zf) 
such that ^^(expy) = e^^^\ y G f)c exists. Let L' C L be the subset of regular elements. 
According to Harish- Chandra the discrete series of G is parametrized by L', i.e., for each 
fi & L' there is a discrete series representation vr^. Moreover vr^ = n^/ iff there exists 
w & W such that p = wp,', and each discrete series representation is of the form vr^ for 
some p G L'. Then by |Arl (6.8)] we have 

(6.19) 7r^(fi) = |p + p|2-|p|2, pGL'. 



tr/i^(5()7r(5() rf^f 



5^ / tTh^{g)rA9) dg 
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So Proposition 16.31 gives in this case 

(6.20) e,^(tr/i^) = ^ (V|yu + p|2- |p|2 + A,) , p e L', TT^ G G{r). 

Using the Plancherel formula, (16.181) and (I6.20p . we get an exphcit form of the trace formula 

(iSD. 

Now we consider the case r = 1, where 1 denotes the trivial representation. Then the h^p 
belongs to the space {G / / K) of iiT-bi-invariant, smooth, compactly supported functions 
on G. Let c(A) be Harish-Cahndra's c-function. Then the Plancherel measure for the 
spherical Fourier transform is given by |c(A)|^^(iA, and the Plancherel formula for spherical 
functions (see |He] ) gives 

(6.21) h^{e) = ^^(a/A2 + |p|2) \c{\)\-^d\. 

Furthermore, note that by Frobenius reciprocity M(l) consists only of the trivial repre- 
sentation 1 of M, and by (16.181) we have 

(6.22) ei,A(tr h^) = ip (VA2 + |p|2) , A G M. 

Inserting (16.101) and (I6.22p into (16.51) we get the final trace formula for the Laplacian on 
functions. If we replace A* by A* — |p|2, then Theorem 11.11 follows. 

7. Restrictions of representations of G 

In this section we consider representations of P which are the restriction of a finite- 
dimensional complex representation r/: G — > GL{E) of G. For such representations there 
exists another approach to the Selberg trace formula. 

Denote the fiat bundle associated to r/lr by -E^. There is a different description of 
as follows. Let E^- = T\Et be the locally homogeneous vector bundle associated to the 
restriction t of rj to K. Then there is a canonical isomorphism 

(7.1) E^ ^ E, 

[MM[ Proposition 3.1]. Note that the space of C°°-sections of Ej. can be identified with 
the space {C°°{T\G) ® E)^ of i^- invariant vectors in {C°°{T\G) O E), where K acts by 
k 1-^ R{k) ® rjlk), k E K. Thus there is a canonical isomorphism 

(7.2) (P:C^{X,E^)^{C^{T\G)®Ef. 

Let = t©p be the Cartan decomposition. By [MMl Lemma 3.1] there exists a Hermitian 
inner product (■, ■)£; in which satisfies the following properties. 

(ry(F)M, v)e = -{u, viY)v)E, for Yet, u,v E E; 

{rj{Y)u, v)e = {u, 'r]{Y)v)E, for Yep, u,v e E. 
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In particular, {■,-)e is -fT- invariant. Therefore, it defines a G-invariant Hermitian fiber 
metric in E^- which descends to a fiber metric in E^-. By (17. ip it corresponds to a fiber 
metric in Er/. Let = (V'')*V'' be the associated Laplacian in C°°{X, E^i). It is a formally 
self-adjoint operator. Its spectral decomposition can be determined as follows. By Kuga's 
lemma [MMt (6.9)] we have 

(7.3) Ar, = -R{n) + 7]{n)id. 

Assume that t] is absolutely irreducible. Then there is a scalar A,, > such that 

r]{n) = A^Id. 

Let Rr be the right regular representation of G in L'^{r\G). Let 

(7.4) L\T\G)='^^^^^mr{7r)H^ 

be the decomposition of Rr into irreducible subrepresentations, where denotes the 
Hilbert space of the representation tt. Denote by (Ti.-,^ ® E)^ the space of K invariant 
vectors of Tij^ (S> E, where the action of K is given by A; i-^ 7c{k) (g) ri{k). By (17. II) and (I7.4p 
we get 

(7.5) L''{X,Er,) = {L\T\G)0Ef^Q ^mr{n){n^ Ef . 
For TT e G let 

A. = vr(n) 

be the Casimir eigenvalue of vr. Then R{Q) acts in (7i^ E)^ by A^^. By (17. 3p it follows 
that w.r.t. the isomorphism (17. 5p . acts in (?i^ ® E)^ as (— A^^ + A^) Id. Thus (17. 5p is 
the eigenspace decomposition of A^. 

Let ip G iS(M) be even. By Lemma [2.41 ip ((A^)^/^) is a smoothing operator. So it is a 
trace class operator. It acts in {'Ht,0E)^'^ by ip ((— A^ + A^)^/^). Then it follows from (17. 5p 
that 

(7.6) Try, ((A,)V2) = ^mr(7r) dim(7^. ® Ef^ ((-A. + X,)'/') . 

To derive the trace formula, we can proceed as in section O The lift A^ of A^ to S 
is a G-invariant elliptic differential operator which is symmetric and non-negative. Let 
hri,ip- T\G End(ii^) be the kernel of (y9((A^)^/^) . Applying Proposition 15.11 with x = 1 
and ( I7.6p . we get 

J2 "^r(vr) dim(H^ ® E)""^ ((-A^ + A^)^/^) = vol(r\^) tr /i^,^(e) 

+ vol(r^\G^) / tr h^^^{g~^-fg) dg. 

Remark. Let x = '7lr- Then we also have the trace formula of Proposition 15. II with r = 1. 
The two formulas are, of course, different, since the operators are different. In the present 
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case, the advantage is that we can work with self-adjoint operators. On the other hand, 
the formula fl5.7p is more suitable for applications to Ruelle- and Selberg zeta functions. □ 

If the split rank of G is 1, we can use fl6.3l) to express the orbital integrals in terms of 
characters. This gives 

Proposition 7.1. Assume that the split rank ofG is 1. Lett]: G GL{E) he an absolutely 
irreducible finite- dimensional complex representation of G. Let ip G be even with 

(p G C^(M). Then with the same notation as above we have 

J2 ^r(7r) dimin^ ® E)^v^ ((-A^ + A^)^/') = vol(r\S) tr /i^,^(e) 
(7 8) ''^^ 

The characters Qa,\{hr,,Lp) can be computed by the method explained in section [Hi 

So there are two classes of finite-dimensional representations of F for which we can work 
with self-adjoint operators and apply the usual Selberg trace formula. These are unitary 
representations and restrictions of rational representations of G. In general, not every rep- 
resentation of r belongs to one of these classes. However, if rank(G) > 2, the superrigidity 
theorem of Margulis |Mal Chapt. VII, §5] implies that a general representation of F is 
not to far from a representation which is either unitary or the restriction of a rational 
representation. See |BW| p. 245] for more details. 
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